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13.  ABSTRACT  (Uwamum  200  wonts) 

Neuroscientists  have  known  for  decades  that  sensory  information  is  encoded  in  the  intervals  between  the  action  potentials 
or  “spikes”  characterizing  neural  firing  events.  Statistical  analyses  of  experimentally  obtained  spike  trains  have  shown  the  ex¬ 
istence  of  a  significant  random  component  in  the  inter-spike  intervals.  'Iliere  has  been  speculation,  of  late,  that  the  noise  may 
actually  facilitate  the  transmission  of  sensory  information;  certainly  there  exists  evidence  that  noise  in  networks  of  neurons  can 
dynamically  alter  the  properties  of  the  membrane  potential  and  the  time  constants  [37,66].  This  paper  describes  the  recent 
rekindling  of  interest  in  the  Stochastic  Resonance  phenomenon  leading  to  speculation  that  such  nonlinear  cooperative  effects 
may  occur  naturally  in  living  systems. 
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1.  INTRODUCTION 

Neuroscientists  have  known  for  decades  that  sensory  information  is  encoded  in  the 
intervals  between  the  action  potentials  or  "spikes"  characterizing  neural  firing  events. 
Statistical  analyses  of  experimentally  obtained  spike  trains  have  shown  the  existence  of  a 
significant  random  component  in  the  inter-spike  intervals.  There  has  been  speculation,  of 
late,  that  the  noise  may  actually  facilitate  the  transmission  of  sensory  information;  cer¬ 
tainly  there  exists  evidence  that  noise  in  networks  of  neurons  can  dynamically  alter  the 
properties  of  the  membrane  potential  and  the  time  constants  [37,66].  The  recent  re- 
kin^ing  of  interest  in  the  Stochastic  Resonance  phenomenon  has  lead  to  speculation  that 
such  nonlinear  cooperative  effects  may  occur  naturally  in  living  systems. 

Stochastic  resonance  (SR)  is  a  cooperative  nonlinear  phenomenon  wherein  the 
signal-to-noise  ratio  (SNR)  at  the  output  of  a  noisy  nonlinear  dynamic  system  driven  by  a 
weak  deterministic  modulation  (which  we  shall  take  to  be  time-periodic),  can  actually  be 
enhanced  by  increasing  the  noise.  Given  a  bistable  dynamic  system,  for  example,  infor¬ 
mation  is  transmitted  through  the  system  in  the  form  of  switching  events  between  the 
stable  states  (attractors)  of  the  potential  function  underlying  the  dynamics.  Suppose  a 
periodic  signal  is  appli^;  its  effect  is  to  rock  the  potential,  alternately  raising  and  lower¬ 
ing  the  wells.  However,  should  its  amplitude  be  very  low  (compared  to  the  height  of  the 
potential  barrier),  it  will  not  be  able  to  induce  switching.  In  the  presence  of  even  small 
amounts  of  noise  (assumed  throughout  this  work  to  be  Gaussian  and  delta-correlated) 
there  will,  however,  always  be  a  finite  switching  probability.  Since  the  switching  proba¬ 
bility  is  greater  when  the  system  is  in  the  ’elevated’  well,  which  occurs  when  the  signal  is 
at  its  maximum,  one  realizes  that  the  noise-induced  switching  events  may  acquire  some 
degree  of  coherence  with  the  deterministic  signal  as  long  as  certain  important  system 
parameters  (notably  the  potential  barrier  height  and  the  locations  of  the  fixed  points)  are 
appropriately  adjusted,  "nie  power  spectrum  obtained  from  the  time-series  solution  of  this 
system  shows  a  sharp  signal,  together  with  its  weaker  odd  harmonics  (for  the  case  of  a 
symmetric  potential),  superimposed  on  a  Lorentzian-like  noise  background.  Theoreti¬ 
cally,  the  signal  is  represented  as  a  8-function  spike;  in  practice,  of  course,  the  spike  has  a 
finite  width  and  amplitude  determined  by  the  bandwidth  of  the  measuring  system  [73,74]. 
Then,  in  order  to  compare  theoretical  results  with  experiments  or  simulations,  one 
integrates  over  the  line-shape  of  the  signal  function  (at  the  fundamental  frequency).  If  the 
signal  feature  is  narrow  and,  in  particular,  falls  entirely  within  one  bin  of  the  (discrete) 
experimental  Fourier  transform  process,  then  one  replaces  the  8-function  by  l/Ao),  the 
inverse  of  the  width  of  a  single  frequency  bin  in  the  FFT.  The  area  obtained  by  the 
integration  represents  the  fraction  of  total  switching  events  that  arc  coherent  with  the 
stimulus.  With  increasing  noise,  the  signal  power  spectral  density,  5  (measured  at  the 
fundamental  frequency  to),  rises  until,  for  a  critical  noise  strength,  the  intra-well  motion 
gives  way  to  inter-well  (or  hopping)  motion  as  the  major  contributor  to  the  dynamics.  At 
this  point  S  decreases  with  noise;  for  very  large  noise  strengths,  the  switching  becomes 
noise-dominated  and  very  rapid,  with  ^1  coherence  with  the  periodic  signal  being 
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destroyed.  For  modulation  frequencies  comparable  to  the  Kramers  rate  (the  characteris¬ 
tic  well-to-well  switching  rate  in  the  noise-only  case),  the  critical  noise  strength  (at  the 
maximum  of  the  signal-power  curve)  corresponds  to  a  matching  between  the  modulation 
frequency  and  twice  the  Kramers  rate,  hence  the  somewhat  misleading  characterization 
of  this  effect  as  a  ’resonance’  in  the  physics  literature.  The  noise  power  spectral  density 
lV((i))  is  given  by  the  background  at  the  base  of  the  signal  feature  and  the  SNR  defined,  in 
decibels,  as  SNR  h  i01og,o(S/(A/  a©)).  A  plot  of  SNR  vs.  noise  stren^  demonstrates  a  bell¬ 
shaped  profile  similar  to  that  of  the  signal  power  spectral  density;  in  the  limit  of  very  low 
(compared  to  the  unperturbed  Kramers  rate)  drive  frequency,  this  curve  and  the  signal 
power  spectral  density  peak  at  approximately  the  same  critic^  noise  strength  [27,50].  In 
the  literatiuo  (and  in  this  review),  the  SNR  vs.  input  noise  variance  profile  is  sometimes 
taken  to  be  the  hallmark  of  SR,  although  the  classical  definition  of  the  resonance  involves 
the  above-mentioned  behavior  of  the  signal  feature  in  the  output,  rather  than  the  SNR. 
The  physics  literature  is  replete  with  theories  and  examples  of  SR  with  good  reviews 
available  in  forthcoming  articles  by  Moss  [52]  and  Jung  [36]  as  well  as  the  proceedings 
of  a  recent  NATO  workshop  on  the  subject  [53].  From  a  biological  perspective,  the 
recent  work  on  SR  in  globally  coupled  arrays  of  nonlinear  switching  elements  [8,35,54] 
is  particularly  significant. 

It  is  important  to  point  out  that  stochastic  resonance  (as  characterized  by  the  bell¬ 
shaped  curve  of  SNR  vs.  noise)  has  not  yet  been  direedy  observed  in  living  systems  with 
internal  noise.  One  of  its  prime  ingredients,  noise-induced  switching,  has  however  been 
demonstrated  in  biological  experiments  [16,24,25].  In  all  such  experiments,  it  is  common 
to  assemble  an  ensemble  of  firing  events  and  fit  a  histogram  to  the  refractory  or  reset 
intervals  occurring  between  the  "spikes".  Such  Inter-Spike-Interval  Histograms  (ISIHs) 
are  ubiquitous  in  the  neurophysiological  literature  and,  as  we  shall  see  below,  can  be  sim¬ 
ply  and  elegandy  explained  by  simple  models  of  neurons  as  bistable  dynamic  switching 
devices  subject  to  noise.  The  results  have  lead  a  few  bold  individuals  to  speculate  on  the 
possible  beneficial  role  of  noise  (in  particular,  cooperative  stochastic  processes  such  as 
described  in  the  preceeding  paragraph)  in  the  processing  of  sensory  neural  information 
[7-11,44-48].  Other  researchers  have  speculated  [40,61,62]  that  the  experimentally 
observable  background  noise  may  be  a  natural  phenomenon  rather  than  a  laboratory 
curiosity.  Statistical  analyses  of  experimentally  obtained  spike  trains  --the  ISIH  is 
obtained  fipom  experiment^  data,  by  assembling  an  ensemble  of  quiescent  intervals  (that 
separate  firing  events)  into  a  histogram-  have  also  shown  the  existence  of  a  significant 
random  component  in  the  inter-spike-intervals.  These  observations  have  lead  neuroscien¬ 
tists  to  ponder  the  role  of  noise  in  sensory  information  processing  for  decades,  usually 
relying  on  experimentally  obtained  spike  trains  and  their  associated  ISIHs  (both  of  which 
arc  ubiquitous  in  the  neuroscience  literature).  It  is  impossible  to  exhaustively  cite  this 
vast  body  of  prior  work,  but  it  must  be  pointed  out  that  our  efforts  complement  and 
extend  conventional  treatments  by  examining  the  response  of  our  model  neuron  to  a 
time-periodic  external  stimulus  that  is  too  small  to  induce  firing  by  itself;  in  this  case,  the 
background  noise  nudges  the  system  past  its  firing  threshold  (which,  as  we  shall  see  in 
the  next  section,  may  actually  be  a  function  of  the  noise).  Hence  the  noise,  in  some  sense, 
mediates  the  response  of  the  model  neuron  to  the  stimulus,  an  idea  that  will  turn  out  to  be 
central  to  our  treatment  of  the  ISIH. 

The  results  of  an  investigation  into  some  aspects  of  the  role  of  noise  in  periodically 
modulated  nonlinear  systems  in  a  neurophysiological  context,  constitutes  the  subject  of 
this  article.  Our  previous  work  [for  a  review,  see  e.g.  refs.  11,12]  consisted  of  modelling 
the  sensory  neuron  as  a  noisy  bistable  (describablc  by  a  dynamics  predicated  on  a  "soft" 
potential)  switching  clement.  Such  a  model  has  been  derived  [8-10]  from  a  fully  coupled 
model  of  a  cell  body  interacting  (at  the  axon  hillock),  vija  a  weak  nonlinear  coupling, 
with  a  dendritic  "bath",  the  dendrites  being  represented  as  quasi-linear  elements  which,  in 
general,  would  also  be  capable  of  "firing".  Of  course,  the  idea  of  bistability  in  this  con¬ 
text  is  not  new;  it  dates  back  to  the  seminal  work  of  Landahl,  McCullough  and  Pitts  [41] 
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who  considered  the  neuron  as  a  discrete  (i.e.  two-state)  noisy  threshold  device.  Today, 
their  model  while  generally  viewed  as  an  over-simplification  of  real  neurons,  has  served 
as  a  convenient  starting  point  for  the  class  of  "connectionist"  models  of  neui^  networks 
[2,17,18,20,30,32,33,49].  Although  our  model  yields  acceptable  fits  to  experimental  data 
and  can  provide  a  very  elegant  explanation  of  the  salient  features  of  experimentally 
observed  ISIHs  (these  are  summarized  in  the  following  section)  it  has  become  evident, 
recently,  that  it  raises  almost  as  many  questions  as  it  answers.  For  instance,  it  is  generally 
accept^  that  neurons  show  bistability,  of  the  type  described  above,  only  in  certain  res¬ 
trictive  regimes  of  parameter  space.  Different  classes  of  neurons  (even  sensory  neurons) 
are  thought  to  exhibit  different  types  of  bistability,  e.g.,  bistability  between  a  fixed  point 
and  limit  cycle  that  characterizes  excitable  oscillators  such  as  those  considered  by  Win- 
free  [73];  the  Fitzhugh-Nagumo  model  of  excitable  cells  [26]  belongs  to  this  class. 
Exjjeriments  show  [24]  that  background  noise  produces  qualitatively  similar  cooperative 
behavior  in  this  class  of  bistable  models  as  well.  Bistability  of  various  kinds  has  been 
quantified  in  simplified  Hodgkins-Huxley-type  models  [23]  and  postulated  via  theoretical 
arguments  [1,57,72].  However,  in  other  simple  quasi-linear  neural  models  e.g,.  the  "per¬ 
fect"  integrate-fire  models,  the  results  can  be  ambiguous.  Here,  noise  introduces  effects, 
in  the  ISM,  that  closely  resemble  those  observed  in  bistable  models;  the  explanation  of 
these  effects  cannot,  however,  be  based  on  the  same  nonlinear  stochastic-dynamic  effects 
that  underlie  the  response  of  bistable  models.  Recent  results  for  this  class  of  models  are 
described  in  section  HI.  Our  fundamental  characterization  of  bistable  model  neurons  as 
threshold  devices  (no  firing  in  the  absence  of  noise)  raises  the  issue  of  information  loss 
during  signal  processing  since  only  the  firing  events,  that  occur  when  a  threshold  is 
crossed,  are  recorded,  with  dynamical  details  regarding  the  passage  to  threshold  being 
ignored.  Of  course,  this  must  be  weighed  against  experimental  studies  that  demonstrate 
the  existence  of  cooperative  stochastic  effects  (e.g.  SR)  that  cannot  be  explained  by  any 
form  of  linear  filtering  theory. 

A  final  caveat:  our  approach  lacks  detailed  "neurophysiological  rigor".  As  physi¬ 
cists,  we  take  a  reductionist  view  by  attempting  to  describe  the  global  response  of  a  com¬ 
plex  system  in  terms  of  the  response  of  a  very  few  degrees  of  freedom  that  characterize 
the  gross  properties  of  the  system  (the  renormalization  group  approach  and  the  enormous 
success  that  it  has  enjoyed  in  treating  complex  systems  is  a  classic  example  of  this  idea). 
As  will  become  evident,  in  the  following  sections,  noise  introduces  qualitatively  similar 
effects  in  different  dynamical  models  of  stimulated  neurons.  These  effects  arc  visible  at 
the  level  of  the  ISM  and  the  SNR,  both  of  which  are  statistical  (i.e.  averaged)  characteri¬ 
zations  of  the  response.  In  addition  to  these  measurements,  other  quantities  that  charac¬ 
terize  the  response  (e.g.  the  mean  firing  rate  and  its  variance)  can  be  obtained  via  simple 
averages  computed  with  respect  to  the  ISM;  these  quantities  can  also  be  directly  meas¬ 
ured  in  repeated  trials  in  an  experiment.  Our  approach  is,  therefore,  quite  adequate  for 
describing  the  coupling  between  the  noise  and  modulation  in  the  neural  dynamics  on  a 
statistical  or  coarse-grained  level,  as  well  as  the  mechanism  whereby  noise  might  actu¬ 
ally  enhance  the  flow  of  information  through  the  system. 

11.  NEURONS  AS  BISTABLE  ELEMENTS  CHARACTERIZED  BY  A  ’SOFT’ 
POTENTIAL;  STATISTICAL  ANALYSIS  OF  SPIKE  TRAINS 

Recent  work  by  Longtin,  Bulsara  and  Moss  (LBM)  [44,45,48]  has  demonstrated 
how  experimental  ISMs  measured,  for  example,  on  the  auditory  nerve  fibers  of  squirrel 
monkey  [60]  could  be  explained  via  a  new  interpretation  of  noise-driven  bistable  dynam¬ 
ics.  They  have  introduced  a  simple  bistable  neuron  model,  a  two-state  system  controlled 
by  a  double-well  potential  with  neural  firing  events  corresponding  to  transitions  over  the 
potential  barrier  (whose  height  is  set  such  that  the  deterministic  stimulus  alone  cannot 
cause  transitions).  The  cell  dynamics  are  described  via  a  variable  xQ),  loosely  denoting 
the  membrane  potential,  and  evolving  according  to 
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x=fix)  +  qsvn<ot+F(t).  (1) 

where  fix)  is  a  hysteretic  flow  function  (expressible  as  the  gradient  of  a  potential  Uix)) 
and  Fit)  is  noise,  taken  to  be  Gaussian,  delta-correlated,  with  zero  mean.  In  the  bistable 
description,  the  potential  is  taken  to  be  the  "soft"  function  Uix)-ax^/2-b  Inicosh  z).  The 
system  G)  has  b^n  numerically  integrated,  with  the  residence  time  in  each  potential  well 
(±ese  times  represent  the  firing  and  quiescent  intervals  in  our  model)  assembled  into  a 
histogram  (the  residence  times  density  function  P(/)),  which  displays  a  sequence  of  peaks 
with  a  characteristic  spacing.  Two  unique  sequences  of  tempoi^  measurements  are  pos¬ 
sible:  the  first  measures  the  residence  time:>  in  only  one  of  the  states  of  the  potential  and 
the  histogram  consists  of  peaks  located  at  t=n  T(/l,  To  being  the  period  of  the  determinis¬ 
tic  modulation  and  n  an  odd  integer.  The  second  sequence  encompasses  measurements  of 
the  total  time  spent  in  both  potential  wells,  i.e.  it  includes  the  active  and  reset  intervals; 
in  the  presence  of  noise,  the  reset  intervals  are  of  largely  stochastic  duration.  The  histo¬ 
gram  corresponding  to  this  sequence  consists  of  peaks  at  locations  i  =  nTo  where  n  is  any 
integer.  The  sequence  of  peaks  implies  a  form  of  phase  locking  of  the  neural  dynamics  to 
the  stimulus.  Starting  from  its  quiescent  state,  the  neuron  attempts  to  fire  at  the  first  max¬ 
imum  of  the  stimulus  cycle.  If  it  fails,  it  will  try  again  at  the  next  maximum,  and  so  on. 
The  latter  sequence  is  the  only  one  observable  in  an  experiment;  the  former  sequence, 
which  corresponds  to  the  refractory  events  is  elegantly  elucidated  by  the  LBM  theory.  In 
addition  to  the  peak  spacing  in  the  ISIH,  most  of  the  other  substantitive  features  of  exper¬ 
imental  ISnis  are  explainable  [47]  via  the  simple  model  (1): 

(a) .  Decreasing  the  noise  intensity  (keeping  all  other  parameters  fixed)  leads  to  more 
peaks  in  the  histogram  since  the  "skipping"  referred  to  above  becomes  more  likely.  Con¬ 
versely,  increasing  the  noise  intensity  tends  to  concentrate  most  of  the  probability  in  the 
first  few  peaks  of  the  histogram. 

(b) .  In  general,  the  probability  density  of  residence  times  is  well  approximated  by  a 
Gamma-like  distribution  of  the  form  PiT)=iT/<T>^)api-TI<T>),  where  <T>  is  the  mean 
of  the  ISIH.  It  is  apparent  that  p(r)-^0  and  cxp(-r/<r>)  in  the  short  and  long  time  limits, 
respectively.  For  vanishingly  small  stimulus  amplitude  q,  the  distribution  tends  to  a 
Gamma,  conforming  to  experimental  observations. 

(c) .  Increasing  the  stimulus  amplitude  leads  to  an  increase  in  the  heights  of  the  lower 
lying  peaks  in  the  ISIH. 

(d) .  Memory  effects  (even  within  the  framework  of  a  description  based  on  the  theory  of 
renewal  processes)  ff^uently  occur,  particularly  at  very  low  driving  frequencies;  they 
manifest  themselves  in  deviations  from  an  exponentially  decaying  envelope  at  low 
residence  times  (the  first  pe^  in  the  ISIH  may  not  be  the  tallest  one). 

(e) .  The  mean  of  the  ISIH  yields  (through  its  inverse)  the  mean  firing  rate. 

(f) .  The  ISIH  decay  rate  X  (the  slope  of  the  envelope  on  a  semi-log  scale)  itself  depends 
sensitively  (exponentially)  on  the  stimulus  amplitude  q  for  constant  noise  strength,  and 
(again,  exponentially)  on  the  noise  strength  for  fixed  q . 

Analog  simulations  of  the  dynamics  yield  an  extremely  good  fit  [44,48]  to  experimental 
dato;  the  fit  can  be  realized  by  changing  only  one  parameter  (the  stimulus  intensity  or  the 
noise  intensity)  and  the  results  are  almost  independent  of  the  functional  form  of  the 
potential  Uix),  depending  critically  on  the  ratio  of  barrier  height  to  noise  variance;  this 
ratio  determines  the  hopping  rate  between  the  basins  of  attraction  in  the  absence  of  noise. 

The  LBM  theory  demonstrates  that  the  peaks  of  the  ISIH  cannot  exist  in  the  absence 
of  noise.  In  fact,  one  could  speculate  that,  over  a  certain  range  of  parameters,  the  noise 
and  signal  play  interchangeable  roles  in  determining  the  shape  of  the  ISIH.  All  the 
stimulus  and  noise  features  arc  encoded  in  the  ISIH,  with  the  phase  preference  (we 
assume  perfect  phase  locking  to  the  stimulus  throughout)  encoded  in  the  peak  width,  this 
assumptions  seems  to  be  consistent  with  experimental  auditory  nerve  data  at  least.  The 
noise  may  be  used  by  the  neuron  to  encode  the  stimulus  features  in  the  ISIH,  while 
preserving  a  fine  amplitude  discrimination  through  the  exponential  dependence  on  X. 
Other  researchers  [66]  have  shown  that  noise  linearizes  the  mean  firing  rate-vs.  current 


-  5  - 


characteristic  in  neurons,  producing  a  large  dynamic  range  along  with  ISIH  variations. 
This  dynamic  range  is  not  found  in  noiseless  Hodgkins-Huxley  type  models.  Identifying 
the  mean  firing  rate  with  the  inverse  of  the  mean  of  the  ISIH  seems  to  lead  one  naturally 
to  this  encoding.  Although  the  LBM  nKxlel  provides  an  imjrortant  first  step  in  the  under¬ 
standing  of  the  (possibly  pivotal)  role  of  noise  in  sensory  information  transfer,  it  is  far 
from  complete.  'Die  residts  do  not  appear  to  depend  critically  on  the  characteristics  of  the 
potential  function  <7(x)  and  the  fundiunental  question:  what  aspects  of  the  data  are  due  to 
the  statistical  properties  of  noisy  two-state  systems  as  opposed  to  real  (i.e.  biological) 
properties  of  cells  that  transcend  this  simple  description,  has  not  been  satisfactorily 
answered. 

Bistable  models  of  the  type  discussed  above  provide  one  of  the  simplest  explana¬ 
tions  of  the  possible  mechanism  underlying  the  processing  and  coding  of  sensory  infor¬ 
mation  in  the  nervous  systenr  They  elucidate  the  sequence  of  reset  events  that  follow 
each  neural  firing,  and  noise  is  seen  to  play  a  pivotal  role  in  the  production  of  the  ISIHs. 
The  dynamics  in  a  generic  bistable  system  of  the  form  (1)  displays  the  N-shaped  flow 
characteristic  that  is  known  to  exist  in  exciteable  cells  [57].  In  our  derivation  of  the 
bistable  model  from  the  fully  coupled  neuro-dendritic  model,  the  coefficients  a  and  b  in 
the  potential  function  U(x)  are  obtained  in  terms  of  the  cell  and  bath  parameters.  Hence, 
the  effect  of  the  dendritic  coupling  is  to  modify  the  height  of  the  potential  barrier, 
thereby  directly  affecting  the  switching  dynamics  and  the  structure  of  the  ISIH.  The 
potential  function  for  the  system  becomes  bimodal  above  a  critical  value  of  b /a  ;  this  tran¬ 
sition  to  bimodality  is  mediated  by  the  noise  in  the  dendritic  bath;  in  fact,  the  theory 
introduces  the  concept  of  a  noise-dependent  firing  threshold  (defined  as  the  barrier 
height).  Stochastic  resonance  (as  described  in  the  preceeding  section)  is  readily  observ¬ 
able  in  these  many-body  models  and  the  output  SNR  can  be  further  enhanced  (over  its 
expected  value  for  the  isolated  case)  by  the  coupling  to  the  dendritic  bath.  Since  none  of 
the  cooperative  effects  that  we  have  described  here  will  occur  in  a  monostable  potential, 
the  critical  role  of  background  noise  is  evident. 

The  physical  model  outlined  above  is  striking  for  its  simplicity  and  provides  an 
elegant  explanation  for  several  features  of  experimentally  computed  ISIHs.  Nonetheless, 
it  can  be  criticized  for  lacking  the  neurophysiological  rigor  afforded  by,  say,  the 
Hodgkins-Huxley  equations  and  their  derivants,  including  the  class  of  "Integrate-and- 
Fire"  models.  We  now  consider  some  of  the  statistical  properties  of  the  simplest  of  these 
models. 

in.  THE  ’PERFECT  INTEGRATOR’  REVISITED 

In  our  work  to  date,  we  have  confined  our  attention  to  nonlinear  dynamic  models  of 
neuronal  firing;  the  models  rely  on  bistable  firing  characteristics  to  model  the  dynamics. 
However,  the  neurophysiological  literature  is  replete  with  stochastic  models  of  spontane¬ 
ous  and  induced  firing  which  are  similar  in  spirit  to  the  bistable  models:  firing  takes  place 
when  the  membrane  voltage  crosses  a  threshold.  However,  the  models  are  not  all  charac¬ 
terized  by  hysteretic  flow  functions.  The  most  ubiquitous  of  these  models,  the  class  of 
integrate-fire  (IF)  models,  can  be  heuristically  derived  from  single-neuron  dynamics  of 
the  form  (1),  in  the  limit  when  the  nonlinear  term  in  the  dynamics  is  very  weak,  so  that 
the  potential  U(x)  is  approximately  parabolic,  using  stochastic  linearization  techniques 
[5,59].  In  the  most  general  case,  the  "leaky"  integrator  with  threshold,  one  writes  the 
dynamics  in  the  form  [65], 

i  =  X.(u,.-x)+p+f  (0+?  coscot ,  (2) 

which  represents  an  Omstein-Uhlenbeck  process  [15,69,71]  in  which  p  is  the  (positive) 
drift  to  a  firing  threshold  a ,  and  X  is  a  decay  constant  governing  the  decay  of  the  voltage 
variable  j:(t)  to  an  equilibrium  or  resting  level  F(r)  is  taken  to  be  white  noise  having 
zero  mean  and  variance  D .  Equation  (2),  in  the  absence  of  the  peritxiic  stimulus,  has  been 
extensively  studied  [3,14,19,21,31,34,39,42,43,55,56,63,67,68]  in  the  theoretical 
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neiux)science  literature.  In  these  studies,  it  is  generally  assumed  that  the  noise  term  F(/) 
represents  the  net  contribution  from  all  the  synaptic  inputs  to  the  cell.  Deterministic  IF 
models  have  been  studied  by  Keener  ct.  al.  [38]. 

We  consider  now  a  simplification  of  the  above  model  due  originally  to  Gerstein  and 
Mandelbrot  (GM)  [29]  in  which  the  ^=0.X;=0  case  was  considered,  the  so-called  "perfect 
integrator".  Assuming  the  underlying  dynamics  to  be  time-stationary,  a  random  walk 
description  of  the  neural  firing  dynamics  has  been  invoked  by  GM,  based  on  the  corner¬ 
stone  requirement  of  a  stable  distribution  function  for  the  probability  density  of  first  pas¬ 
sage  times  corresponding  to  the  dynamics.  The  state  variable  x  is  assumed  to  execute  a 
biased  random  walk  to  an  absorbing  threshold  at  which  point  a  firing  event  is  designated 
to  have  occurred  and  the  membrane  potential  x  is  then  instantaneously  reset  to  its  starting 
value  (the  reset  mechanism  being  purely  deterministic).  It  is  this  reset  which  renders  the 
’global’  dynamics  nonlinear.  The  distance  between  the  origin  and  the  threshold  is  the 
"barrier  height"  a,  analogous  to  the  height  C/q  of  the  potential  barrier  in  our  bistable 
model.  Further,  it  is  assumed  that  the  motion  in  phase  space  occurs  under  the  influence 
of  a  positive  drift  coefficient  p  which  was  defined  by  GM  as  roughly  the  difference 
between  the  drift  velocities  corresponding  to  excitatoiy  and  inhibitory  synaptic  inputs  (it 
is  neurophysiologically  reasonable  to  assume  these  velocities  to  be  different).  For  this 
model,  the  Wiener  process  with  drift,  one  readily  writes  down  the  associated  Fokker 
Planck  Equation  (FPE)  [28,58,70]: 


dx  2  dx^  ’ 


(3) 


whence  the  probability  density  function  of  first  passage  times  is  [22,29], 


go(0= 


(a  -ap-pu)^' 
2Dt 


(4) 


Go  being  the  staning  point  of  the  random  walk.  The  density  function  ^o(0,  often  referred 
to  in  the  statistical  literature  as  the  ’Inverse  Gaussian’,  reproduces  many  of  the  properties 
of  experimentally  observed  ISIHs  for  the  spontaneous  finng  case.  The  mean  first  passage 
time  (a-ao)/M-  to  the  absorbing  threshold  is  calculated  as  the  first  moment  of  g^t),  and  its 
reciprocal  yields  an  average  firing  rate.  The  variance  of  the  first  passage  time  is 
D  {a-a^iv?.  The  model  reproduces  the  salient  features  of  the  ISIHs  obtained  from  spon¬ 
taneous  fmng  events  reasonably  well,  including  the  long  (exponential)  tail  in  the  first 
passage  time  density  function  g{t).  It  also  has  numerous  limitations  (which  have  been 
ennumerated  by  GM),  the  most  fundamental  one  being  the  assumption  that  the  mem¬ 
brane  potential  is  reset  following  each  spike,  with  the  random  walk  to  the  threshold  re¬ 
starting;  nevertheless,  the  model  provides  a  simple  vehicle  to  explain  the  dynamics  (on  a 
grossly  simplified  level)  that  arise  from  the  coupling  of  the  noise  and  the  periodic 
stimulus.  A  more  rigorous  (from  a  neurophysiological  standpoint)  grounding  of  the  FPE 
(3)  has  been  given  by  Stevens  [64].  In  order  to  make  even  better  contact  with  experimen¬ 
tal  results,  it  is  necessary  to  provide  reasonably  good  numerical  values  for  the  drift 
coefficient  p,  the  "barrier  height"  a  and  the  background  noise  variance  D  in  the  above 
model.  A  first  attempt  to  do  so  (while  simultaneously  providing  a  test  of  the  goodness  of 
fit  of  the  model  to  neurophysiological  data)  was  carried  out  by  Berger  et.  al.  [4],  They 
carried  out  an  experiment  aimed  at  recording  the  inter-spike-interval  distribution  from 
extra-cellular  recordings  on  the  cat  visual  cortex.  Having  obtained  the  experimental 
ISIHs,  they  were  able  to  compute  the  equivalent  model  quantities  p  and  a  via  the  mean 
and  stand^  deviation  of  the  experimentally  obtained  ISIHs,  assuming  a  fixed  back¬ 
ground  noise  variance  D .  While  we  do  not  give  any  further  details  of  the  experiment,  it  is 
noteworthy  that,  once  these  "self-consistent"  values  of  p,  a  and  D  were  substituted  into 
the  first  passage  time  probability  density  function,  an  excellent  fit  of  the  model  to  the 
experimental  ISIHs  resulted. 


We  now  consider  the  GM  nxKlel  in  the  presence  of  a  deterministic  periodic  stimulas 
q  cos  (lit,  assuming  the  initial  condition  ao=0.  Once  again  we  assume  the  phase-locked 
case.  One  can  then  write  down  and  solve  [13]  the  FPE  to  yield  the  probability  density 
function  The  first  passage  times  density  function  is  then  computed  via  the 

prescription  [22]: 

which  yields,  after  some  calculation, 

g (0  =  ^  ^ ~  a  ^  ***' 

where  <t>,.(x)s  i  -  f  e'^’di  is  the  Complementary  Error  function  and  we  have  defined. 
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We  now  discuss  briefly  some  of  the  propenies  of  the  density  functions  (4)  and  (5). 
A  plot  of  (5)  yields  an  ISIH  with  the  peaks  at  locations  nT^  where  n  is  an  integer  and  To 
the  stimulus  period.  The  hidden  or  reset  mechanism  in  this  model  is  instantaneous  (and 
deterministic),  in  contrast  to  the  bistable  models.  However,  the  model  differs  from  the 
bistable  models  in  another  very  important  respect.  In  the  absence  of  noise  and  the 
periodic  stimulus,  the  state-point  will  reach  the  threshold,  provided  the  drift  p  is  positive; 
the  model  admits  of  the  possibility  of  deterministic  ’switching’  or  firing.  For  the  <7«p 

case,  a  first-order  perturbation  solution  (for  t)  of  the  dynamic  equation  a  =p/  +  ^sina)t 

yields  the  deterministic  passage  time. 


sin  oMo 
aci) 


cos  Wo 


(6) 


where  tQ=ai\i  is  the  mean  first  passage  time  (MFPT)  in  the  presence  of  noise  alone.  Note 
that  T  to  2d.  high  drive  frequencies  co,  as  should  be  expected.  The  deterministic  passage 
time  (6)  is  also  the  MFPT  in  the  presence  of  the  noise  and  the  periodic  stimulus.  For  a 
long-tailed  distribution  of  the  form  (3)  however,  possibly  the  more  important  quantity  is 
the  most  probable  time,  or  the  mode.  For  the  ^=0  case,  this  time  is  readily  obtained  by 
differentiation: 


It  is  significant  that  the  mode  t„  depends  on  the  noise.  In  experiments,  firing  times 
clustered  about  the  mode  are  more  probable.  The  mean  of  a  large  number  of  firing  times 
may  yield  a  MFPT  ro  which,  depending  on  the  physical  characteristics  of  the  density 
function  (3)  is  close  to  the  mode  or  far  out  in  the  tail.  When  analysing  the  properties  of 
the  ISIH  the  interplay  of  the  three  time-scales  To, to.  and  tm  is  crucial.  With  decreasing 
inhibition  (represented  as  increasing  drift  p  for  fixed  excitation)  or  with  decreasing  noise 
strength  (keeping  the  drift  fixed),  the  ISIH  approaches  a  more  sharply  peaked  (Gaussian- 


s 

like)  density  (in  effect,  the  tail  of  the  Gamma-like  density  (3)  shrinks,  with  the  mean 


Fig  1.  Mode  of  the  noise-only  density  function  (4),  normalized  to  the 
mean,  vs.  the  drift  coefficient  u  for  a  =20  and  noise  variance 


D  =0.1,0.25,0,5  (top  to  bottom  curves). 


Fig  2.  Mode  of  the  noise  only  density  function  (4),  normalized  to  the 
mean,  vs.  the  noise  variance  D  for  <7=20  and  drift  p=0.l ,  0.065 ,0.025 
(top  to  bottom  curves). 
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approaching  the  This  may  be  demonstrated  via  a  Gram-Charlier  expansion  of  the 

density  function  >T)  and  is  depicted  in  figures  1  and  2  in  which  we  plot  the  ratio  i^/to  vs. 
the  drift  \i  and  fhe  noise  variance  D  respectively.  Increasing  the  drift  leads  to  a  more  shar¬ 
ply  peaked  distribution  (characterized  by  the  rate  at  which  r«//o-^  0  for  a  given  noise 


Fig  3.  First  passage  time  density  function  (5)  and  its  <7=0  counterpart  (4) 
for  (a  .a).D)  =  (20,.04..03..1  ..2). 


Fig  4.  Same  as  figure  3  with  p=0.065. 
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variance;  the  approach  to  a  very  sharply  p)cakcd  distribution  is  more  rapid  at  lower  noise 
strengths  as  seen  in  both  figures.  Figures  3-5  show  the  effect  of  increasing  the  drift  ^  on 
the  ISIH;  in  these  figures,  the  smooth  curves  represent  the  density  function  (3).  Note 
that  if  we  introduced  a  phase  into  the  argument  of  the  periodic  stimulus  and  then 
defined  a  phase-averaged  ISEH  (the  averaging  being  perform^  over  a  uniform  distribu¬ 
tion  on  (0,231]),  the  two  curves  in  each  of  the  figures  3-5  would  coincide;  for  the  phase- 
locked  case,  one  obtains  an  ISIH  that  is  qualitatively  similar  to  those  observed  for  the 


Fig  5.  Same  as  figure  3  with  u=0.l. 


bistable  models.  The  mean,  variance,  and  mode  of  the  ISIH  (for  small  q)  are  seen  to  be 
extremely  close  to  the  corresponding  quantities  for  the  q=0  ISIH,  as  preicted  above.  It 
should  be  evident  that  if  we  adjust  the  stimulus  and/or  system  parameters  such  that 

lm=nTo.  (8) 

then  the  n'*-  peak  in  the  ISIH  will  be  the  highest  one. 

The  interesting  feature  about  the  ISEH  (5)  is  that  the  peak  heights  go  through  max¬ 
ima  as  functions  of  the  noise  variance  D ,  all  other  parameters  being  held  constant.  This  is 
demonstrated  in  figure  6  for  a  particular  set  of  system  and  modulation  parameters.  One 
can  calculate  from  (5)  the  critical  noise  D,  at  which  the  n**  peak  passes  through  its  max¬ 
imum.  Note  that  the  peaks  in  (5)  occur  at  times  i=nTo.  Hence,  the  amplitude  of  the  n"' 
peak  is  given  by 

(9) 
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(10a) 

(10b) 


where 


arc  the  contributions  from  the  ^=0  and  ^>0  parts  of  the  density  function  (7)  (we  set 
t.  =nro=2ii  jt/co).  Writing  =D^o+5  where  D*o  is  the  noise  vari^ce  at  which  the  expres-  - 
sion  (10a)  is  maximized  and  6  is  the  (much  smaller)  contribution  from  (10b),  we  easily 
find 

(13) 

The  remaining  contribution  6  is  found  by  setting  the  derivative  of  (9)  equal  to  zero  and 
then  expanding  to  0(5).  This  contribution  is.  however,  much  smaller  than  and  can  be 
neglected  for  the  present.  In  figure  7,  we  focus  our  attention  for  a  particular  peak  (n=4). 
We  set  pep,  =0.159  where,  for  this  case.  p«  is  found  by  setting  nTo=to-  For  a  fixed  noise 
variance  D ,  this  value  of  p,:  maximizes  the  height  of  this  peak,  although  this  does  not 
imply  that  the  peak  in  question  is  the  highest  one  in  the  ISIH.  Now,  the  noise  variance  is 
further  decreased.  This  has  the  effect  of  moving  the  mode  such  that  the  n**  peak  coin¬ 
cides  with  the  mode  (/„  =nTo)  and  this  peak  becomes  the  highest  one.  Decreasing  the 
noise  further  causes  the  peak  to  get  taller  and  narrower,  it  becomes  delta-function-like 
and  attains  its  maximum  height  in  the  to limit  as  discussed  earlier. 


Fig  6.  Peak  heights  vs.  noise  variance  D  for  the  density  function 
(5)  with  (a  .p,<7  .(o)e (20,. 055,. 0475).  Curves  correspond  to 
(reading  from  left  to  right)  peak  numbers  «  =7.6.5.4,3. 


IV.  DISCUSSION,  CONCLUSIONS  AND  SPECULATION 

It  seems  fitting  to  conclude  this  article  with  a  question  and  some  speculation. 
Clearly  the  question  to  be  answered  is:  "Clan  the  neuron  indeed  be  characterized  by  a 
noisy  nonlinear  switching  element,  describeable  by  a  bistable  dynamics  of  the  form  (1) 
(with,  say,  a  ’soft’  potential  U(x))  or  a  far  simpler  switching  element  described  by  the 
dynamics  (2)?"  C^nainly,  there  is  ample  evidence  that  noise  plays  a  critical  role  in  the 
transmission  and  coding  of  information  in  the  nervous  system;  most  reasonable  models 
of  neural  firing  invoke  some  kind  of  diffusion  process  representation  of  the  system  on  a 
statistical  or  coarse-grained  level  (in  which  the  system  is  described  via  a  probability  den¬ 
sity  function  rather  than  an  individual  trajectory  solution).  It  seems  reasonable  then,  that 
there  should  exist  a  form  of  self-regulatory  mechanism  such  that  the  internal  parameters 
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of  the  system  (these  parameters  control,  for  instance,  the  height  of  the  potential  bar¬ 
rier)  can  be  adjusted  in  response  to  the  stimulus  and  noise  characteristics.  In  our  previous 


Fig  7.  Height  of  peak  no.  4  with  (a  .g  .&))h(20..0475,.2)  and 
P=0.159  (critical  value.. .see  text).  Top  curve:  £>=0.012,  middle  curve 
D=0.06,  bottom  curve:  0=1.0. 

work  [8,9],  in  fact,  we  have  been  lead,  in  a  simple  way  to  the  concept  of  a  noise- 
dependent  firing  threshold.  This  is  important  because,  for  given  stimulus  and  noise,  one 
retains  well-defined  histograms  for  only  a  small  range  of  Uq-  One  may  speculate  further 
that  the  neuron  (or  the  network)  then  uses  the  bcckground  noise  so  that  its  response 
(measured  via  the  SNR  or,  cquiv^enily,  through  the  ISIH)  is  optimized.  This  implies 
that,  for  configurations  that  admit  of  stochastic  resonance,  the  neuron  (or  network) 
operates  close  to  the  maximum  of  the  stochastic  resonance  curve  (SNR  vs.  noise  vari¬ 
ance)  while  simultaneously  obtaining  other  information  (e.g.  amplitude,  frequency  and 
phase)  about  the  deteiministic  stimulus,  via  the  ISIH.  In  effect,  our  construction  and 
interpretation  of  the  ISIH  (together  with  the  remarkable  ability  to  explain  most  of  the 
features  of  experimentally  obtained  ISIHs)  as  a  natural  outcome  of  our  modelling  the 
neuron  as  a  noisy  bistable  switching  clement  (in  the  LBM  theory)  or  even  via  the 
extremely^  simple  model  of  section  III  implies  that  sensory  neurons  may  measure  the 
stimulus  intensity  by  comparing  it  to  the  background  noise,  using  the  (internally 
adjusted)  potential  barrier  height  to  mediate  and  optimize  the  measurement  [F  Moss 
1992;  pnvatc  ^scussipnsj.  In  the  perfect  integrator  model,  the  firing  threshold  is 
^uivalent  to  a  ’bamcr’  in  the  jargon  of  bistable  dynamics;  such  models  may  be,  in  fact, 
derivable  from  nioie  complex  models  such  as  those  considered  in  the  LBM  studies.  Our 
studies  of  collective  behavior  in  large  networks  show  that  the  coupling  to  other  elements 
degrade  the  SNR  depending  on  the  magnitudes  and  signs  of  the  coupling 
coefficients,  i.c.,  the  excitatory  or  inhibitory  nature  of  the  interactions  is  critical.  In  the 
perfect  integrator,  the  drift  p  is  controlled  by  the  inhibition  which,  therefore,  plays  a 
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critical  role  in  even  this  simplest  of  descriptions  of  neural  dynamics.  What  should  be 
amply  clear  from  this  study  and  our  previous  work  is  that  there  is  no  single  model  that 
describes  the  dynamics  of  neurons  everywhere  in  the  nervous  system;  even  the  very  sim¬ 
ple  model  of  section  HI  has  its  drawbacks,  but  it  serves  to  provide  a  simple  description  of 
the  role  of  noise  and  inhibition  in  the  firing  dynamics  of  a  single  cell.  In  fact,  a  generali¬ 
zation  of  this  model,  the  so-called  ’leaky’  integrator  characterized  by  the  full  dynamics 
(2)  may  well  provide  a  good  model  of  the  bursting  that  is  so  ubiquitous  a  feature  of 
axonal  firing. 

The  Inter-Spike-Interval  Histograms  arc  not,  by  themselves,  necessarily  an  indicator 
of  the  presence  of  stochastic  resonance  as  an  underlying  cooperative  effect  in  neurophy¬ 
siology.  For  bistable  models  having  the  general  form  (1)  they  are,  however,  a  product  of 
correlated  (between  the  noise  and  deterministic  modulation)  switching  and,  as  we  have 
indicated  in  section  I,  there  is  some  evidence  that  such  switching  processes  do  occur  in 
neurophysiology  (although  the  form  of  the  bistabUity  may  be  different  from  the  potential 
systems  defined  by  dynamics  such  as  (1)).  Various  features  of  these  histograms  can,  how¬ 
ever,  lend  themselves  to  explanations  based  on  stochastic  resonance.  Perhaps  the  most 
important  of  these  features  is  that  the  heights  of  successive  peaks  pass  through  a  max¬ 
imum  as  a  function  of  the  noise  strength.  This  has  been  demonstrated  in  section  HI  for 
the  simple  ’perfect’  integrator  model  and  is  also  known  to  occur  in  bistable  systems  of 
the  form  (1)  [47,76].  So  far,  attempts  to  quantify  this  "resonance"  as  a  matching  of  two 
characteristic  rates  have  been  inconclusive,  for  the  bistable  case,  largely  because  of  the 
difficulty  of  (numerically)  producing  good  ISIHs  with  low  noise.  For  the  simple  IF  model 
of  section  III,  however,  this  connection  (or  lack  of  it)  might  be  possible  to  establish,  due 
to  the  analytical  tractability  of  the  problem;  this  is  an  area  currently  being  actively  inves¬ 
tigated.  Since  these  models  admit  of  a  deterministic  switching  (or  firing)  mechanism,  in 
contrast  with  the  fundamental  precept  of  no  deterministic  switching  that  underlies  the 
bistable  models,  the  mechanism  for  the  observed  "resonances"  of  figure  6  may  well  differ 
from  the  corresponding  mechanism  in  the  bistable  case. 

What  should  be  clear  from  this  paper  is  that  effects  that,  qualitatively  at  least, 
appear  to  be  similar  can  occur  in  systems  that  are  quite  different  A  simple  observation  of 
the  IS  EH  or  the  resonance  in  the  peak  heights  as  a  function  of  noise  is  not  sufficient  to 
establish  a  model  for  the  underlying  dynamics.  In  different  terms,  it  is  probable  that  sim¬ 
ple  models  of  the  type  discuss^  in  section  III  may  provide  a  good  fit  to  experimental 
ISIHs  just  as  we  have  seen  [24,44,45,48]  in  various  forms  of  bistable  descriptions.  In 
fact,  we  have  shown  that,  for  the  bistable  case,  a  good  fit  to  experimental  data  is  obtained 
using  dynamics  of  the  form  (1)  with  different  flow  functions  /(x);  the  data  shed  no  light 
on  Ae  precise  form  of  the  dynamics.  This  is  a  characteristic  of  our  coarse-grained 
description  of  the  dynamics  in  terms  of  probabilities  and  power  spectra.  In  most  theories 
of  stochastic  resonance  in  bistable  systems,  the  quantity  that  critically  mediates  the 
dynamics  is  the  ratio  of  the  barrier  height  to  Ae  noise  variance.  This  ratio,  in  fact,  deter¬ 
mines,  as  outlined  in  section  II,  the  number  and  sizes  of  the  peaks  in  the  ISEH.  So,  one 
must  end  this  review  with  the  (by  now)  somewhat  obvious  question:  "Wfliich  features  of 
the  ISIHs  are  due  to  underlying  neurophysiological  processes  and  which  features  arc 
influenced  by  the  statistics  of  the  detection/measurement  process  or  the  statistics  of  the 
simple  models  such  as  (1)  and  (2)  which  have  been  invoked  to  explain  the  features  of  the 
ISIHs?" 
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